What's new in Julia 0.4, and coming in 0.5

Tony Kelman, @tkelman


In [1]:
let cachefile = joinpath(Base.LOAD_CACHE_PATH[1], "DataStructures.ji")
    isfile(cachefile) && rm(cachefile)
end
tic()
using DataStructures
toc();


INFO: Precompiling module DataStructures...
elapsed time: 10.653973631 seconds

In [1]:
# restart kernel
tic()
using DataStructures
toc();


elapsed time: 0.227078362 seconds

To use in a package, add

isdefined(Base, :__precompile__) && __precompile__()

before the first module in main package source file. If you don't need to support 0.3, then you don't need the isdefined.

Test carefully first, some things like saving pointers in global variables, or using eval don't always work when precompiled.


In [1]:
# (restart kernel again)
# automatically recompiles if package source changes
touch(Pkg.dir("DataStructures","src","DataStructures.jl"))
tic()
using DataStructures
toc();


INFO: Recompiling stale cache file C:\Users\Tony\.julia\lib\v0.4\DataStructures.ji for module DataStructures.
elapsed time: 10.371396361 seconds

2. New garbage collector (https://github.com/JuliaLang/julia/pull/8699)

Code that does a lot of intermediate allocation should be much faster now.


In [2]:
A = map(BigInt, rand(1:10, 50, 50));
@time A^50;
@time A^50;
@time A^50;


  1.508227 seconds (8.39 M allocations: 184.221 MB, 8.78% gc time)
  0.607835 seconds (7.26 M allocations: 131.684 MB, 18.09% gc time)
  0.603703 seconds (7.26 M allocations: 131.684 MB, 18.74% gc time)

Under 0.3:

elapsed time: 2.055229508 seconds (152561404 bytes allocated, 22.03% gc time)
elapsed time: 1.159305999 seconds (117920640 bytes allocated, 31.78% gc time)
elapsed time: 1.305996277 seconds (117920640 bytes allocated, 42.00% gc time)

3. Call overloading (https://github.com/JuliaLang/julia/pull/8712)

Arbitrary types can be called like functions, not just their constructors


In [ ]:
UInt(5)
# instead of the now-deprecated
uint(5)

In [3]:
# often used for "functor types" to help with dispatch and inlining, but subject to change again in 0.5
# see Jeff Bezanson's JuliaCon and Aug 2015 BAJU talks
immutable MultiplierType
end
Base.call(::Type{MultiplierType}, a, b) = a * b
map(MultiplierType, 1:5, 6:10)


Out[3]:
5-element Array{Int64,1}:
  6
 14
 24
 36
 50

4. @generated (formerly known as staged) functions (https://github.com/JuliaLang/julia/pull/7474)

See Jake Bolewski's JuliaCon talk for more details https://www.youtube.com/watch?v=KAN8zbM659o

Conventional macros take expressions as input and return an expression output at parsing time.

Generated functions take the types of the inputs, after type inference, and return an expression.


In [4]:
macro examplemacro(foo)
    @show foo
    return :(2 * $foo)
end
@examplemacro (5 * 9 + 10);


foo = :(5 * 9 + 10)

In [5]:
ans


Out[5]:
110

In [6]:
@generated function examplegen(foo)
    @show foo
    if foo == Int
        return :(10)
    else
        return :(15.0)
    end
end
examplegen(5);


foo = Int64

In [7]:
ans


Out[7]:
10

In [8]:
examplegen(1.0);


foo = Float64

In [9]:
ans


Out[9]:
15.0

In [10]:
# generated functions cache their results each time they are called on a given set of input types!
examplegen(1.0) # returns cached result, does not @show Float64 again


Out[10]:
15.0

5. Documentation system, Andy Hayden just talked about it

Docstrings now shown in REPL help for package and user code


In [11]:
"My awesome new function"
yaydocs(a) = 1.0


Out[11]:
yaydocs (generic function with 1 method)

In [12]:
?yaydocs


search: 
Out[12]:

My awesome new function

yaydocs

6. Performance and usability improvements to multidimensional AbstractArrays (https://github.com/JuliaLang/julia/issues/7941 and https://github.com/JuliaLang/julia/pull/8432 and https://github.com/JuliaLang/julia/issues/8501 and https://github.com/JuliaLang/julia/pull/10525)

  • Fast array views built on @generated functions.
  • eachindex iterator for fast Cartesian indexing.
  • Arbitrary AbstractArray types just need to implement size and one of the following getindex methods, then all other combinations of vector, slice, and other "fancy" indexing are handled automatically.
    getindex(::T, ::Int) # if linearindexing(T) == LinearFast()
    getindex(::T, ::Int, ::Int, #=...ndims(A) indices...=#) if LinearSlow()

7. Nullable types (https://github.com/JuliaLang/julia/issues/8152)

Parametric wrapper around a type with a boolean flag to indicate missing data.

Allows type stable operation in the presence of missing data (see John Myles White's talk).

8. Tuple type improvements (https://github.com/JuliaLang/julia/pull/10380)

Tuples of values (a, b, c) now completely separate from types of tuples, Tuple{Int, Float64, ASCIIString}.

Faster implementation, value tuples very useful now as immutable containers.

9. Better interoperability between Julia types and C structs (https://github.com/JuliaLang/julia/pull/2818 and https://github.com/JuliaLang/julia/pull/7906)

Noticeable if you use ccall a lot, new Ref{T} syntax for by-reference passing.

Ongoing work on master for 0.5

1. LibGit2-based package manager rewrite (https://github.com/JuliaLang/julia/pull/11196)

Uses embedded C libgit2 library instead of shelling out to command-line git. Keeps much more information in memory, needs less IO for Pkg operations, much faster.

2. Codegen rewrite (https://github.com/JuliaLang/julia/pull/11973)

Not user visible, but improves maintainability and internal structure of core Julia-LLVM interfaces.

3. New unit test system (https://github.com/JuliaLang/julia/pull/13062)

Extends Base.Test with some features that are currently found in FactCheck.jl (http://github.com/JuliaLang/FactCheck.jl).

  • Grouping tests in labeled test sets
  • Run all tests to completion and provide summary when finished, instead of failing right away

4. Multithreading (https://github.com/JuliaLang/julia/pull/13410)

Simple single-node parallelism via @threads macro.

Will need a non-default build flag at first, depends on newer version of LLVM.

5. LLVM upgrade (https://github.com/JuliaLang/julia/issues/9336)

Needed for multithreading, Cxx.jl, Gallium.jl debugger, etc.

Better runtime performance and vectorization support.

Several patches and more work still needed to maintain debug info, compile time performance, and memory consumption.

6. Major breaking array changes (https://github.com/JuliaLang/julia/issues/13157)

Slices as views, transpose type, dropping dimensions indexed with a scalar